Image pickup apparatus, method for controlling the image pickup apparatus, and recording medium

ABSTRACT

An image pickup apparatus includes a receiving unit configured to receive a designation of an image pickup direction of an image pickup unit in relation to superimposition of a mask image on an picked-up image; and a setting unit configured to set, when the receiving unit receives the designation of the image pickup direction, a position of superimposition of the mask image in an image pickup range where the image pickup unit can pick up images, on the basis of the designated image pickup direction.

CROSS-REFERENCE TO RELATED APPLICATIONS

This application is a Continuation of International Patent ApplicationNo. PCT/JP2013/063244, filed May 13, 2013, which claims the benefit ofJapanese Patent Application No. 2012-115762, filed May 21, 2012, both ofwhich are hereby incorporated by reference herein in their entirety.

TECHNICAL FIELD

The present invention relates to an image pickup apparatus capable ofdelivering a picked-up image, and particularly relates to the functionof superimposing a mask image on a picked-up image picked up by theimage pickup apparatus.

BACKGROUND ART

Conventionally, a mask processing function has been provided, which isfor preventing a specific area of a picked-up image from beingidentified by a delivery destination of a picked-up image. Examples ofthe mask processing function include the function of filling in a targetarea in an on-screen display (OSD), the function of lowering the imagequality of a target area, and the function of filtering.

For example, Patent Literature (PTL) 1 discloses a technique whichinvolves lowering the image quality of a capturing prohibited area inimage data output from an image pickup unit.

A camera described in PTL 1 is connected via a network to a recordingdevice. The recording device designates a capturing prohibited positionin a picked-up image input from the camera. The camera sets a capturingprohibited area at the position designated by the recording device.

CITATION LIST Patent Literature

PTL 1 Japanese Patent Laid-Open No. 2005-323007

The implementation and interface of a mask processing function variesdepending on the camera and the client application.

In a conventional method, a mask position is identified using acoordinate system shared by a camera and a client. In this case, theclient needs to replace mask position information input by a user or thelike with a coordinate value in the coordinate system shared by thecamera and the client, and then to transmit the coordinate value to thecamera. Also, to configure the mask setting for a plurality of camerashaving different coordinate systems, the client needs to designate acoordinate position of a mask image in accordance with the coordinatesystem of each camera. This makes the processing in the clientcomplicated.

SUMMARY OF INVENTION

To solve the problems described above, an image pickup apparatusaccording to the present invention includes an image pickup unit capableof changing an image pickup direction; a transmitting unit configured totransmit a picked-up image picked up by the image pickup unit to areceiving device; a receiving unit configured to receive a designationof the image pickup direction of the image pickup unit in relation tosuperimposition of a mask image on the picked-up image; and a settingunit configured to set, when the receiving unit receives the designationof the image pickup direction, a position of superimposition of the maskimage in an image pickup range where the image pickup unit can pick upimages, on the basis of the designated image pickup direction.

Further features of the present invention will become apparent from thefollowing description of exemplary embodiments with reference to theattached drawings.

BRIEF DESCRIPTION OF DRAWINGS

FIGS. 1A to 1C are external views of a system.

FIGS. 2A and 2B are block diagrams.

FIG. 3 illustrates a parameter configuration.

FIG. 4 illustrates a command sequence.

FIG. 5 illustrates a command sequence for mask setting.

FIG. 6 is a flowchart of mask parameter processing.

FIG. 7 is a flowchart of mask parameter processing.

FIGS. 8A and 8B illustrate a coordinate calculation.

FIG. 9 illustrates a coordinate calculation.

DESCRIPTION OF EMBODIMENTS

Embodiments of the present invention will now be described.

First Embodiment

FIG. 1A illustrates a surveillance camera according to an embodiment ofthe present invention. Reference numeral 1101 denotes a mechanism thatmoves an image pickup unit in a pan direction, reference numeral 1102denotes a mechanism that moves the image pickup unit in a tiltdirection, and reference numeral 1103 denotes a zoom mechanism.

FIG. 1C illustrates a system configuration including a surveillancecamera 1000. Reference numeral 2000 denotes a client device serving asan external device in the present invention. The surveillance camera1000 and the client device 2000 are connected to be able to communicatewith each other via an IP network 1500. The client device 2000transmits, to the surveillance camera 1000, various commands (describedbelow) for changing image pickup parameters, driving a camera platform,and starting video streaming. The surveillance camera 1000 transmitsresponses to the commands and video streaming to the client device 2000.The description of FIG. 1B will be given later on.

FIG. 2A illustrates an internal configuration of the surveillance cameraaccording to the present embodiment.

In FIG. 2A, reference numeral 1001 denotes a control unit that controlsan overall operation of the surveillance camera 1000. For example, thecontrol unit 1001 is a computer formed by a CPU.

Reference numeral 1002 denotes a storage unit (memory). The storage unit1002 is primarily used as a storage area for storing various types ofdata. For example, the storage unit 1002 is used as a storage area forstoring programs executed by the control unit 1001, a work area forprogram execution, or a storage area for storing image data generated byan image pickup unit 1003 (described below).

Reference numeral 1003 denotes an image pickup unit. The image pickupunit 1003 converts an analog signal acquired by capturing an image of anobject into digital data, compresses the data using adaptive discretecosine transformation (ADCT) or the like to generate image data of apicked-up image, and outputs the picked-up image to the storage unit1002. After outputting the picked-up image to the storage unit 1002, theimage pickup unit 1003 issues an image acquisition event to the controlunit 1001. An image pickup direction of the image pickup unit 1003 canbe changed by an image pickup control unit 1006 (described below).

Reference numeral 1004 denotes a communication unit. The communicationunit 1004 is used to receive various control commands from an externaldevice, and to transmit a response to each control command to theexternal device (client device). The received commands include, forexample, a designation of the image pickup direction of the image pickupunit 1003 in relation to superimposition of a mask image on a picked-upimage. The communication unit 1004 transmits, as a delivery image, apart of or the entire picked-up image picked up by the image pickup unit1003 to a receiving device (client device).

Reference numeral 1006 denotes an image pickup control unit. The imagepickup control unit 1006 is used to control the pan mechanism 1101, thetilt mechanism 1102, and the zoom mechanism 1103 in accordance withvalues of a pan angle, a tilt angle, and a zoom factor input from thecontrol unit 1001. That is, the image pickup control unit 1006 canchange the image pickup direction of the image pickup unit 1003. Inresponse to a query from the control unit 1001, the image pickup controlunit 1006 supplies the current pan angle value, tilt angle value, andzoom factor value to the control unit 1001. Also, the image pickupcontrol unit 1006 identifies the current image pickup direction of theimage pickup unit 1003.

Reference numeral 1007 denotes a mask processing unit. The maskprocessing unit 1007 applies a mask to an appropriate position in imagedata output by the image pickup unit 1003. This mask processing is basedon a pan angle, a tilt angle, and a zoom factor input by the controlunit 1001 to identify a mask position. Additionally, this maskprocessing is based on an image cutout position in an image pickup rangeof the image pickup unit 1003 in the pan, tilt, and zoom conditions, andthe designation of the size and position of the mask at the image cutoutposition. The mask processing unit 1007 has an effective flag for eachmask. When the flag is False, the mask processing unit 1007 performscontrol not to output the mask.

An internal configuration of the surveillance camera 1000 has beendescribed with reference to FIG. 2A. However, the processing blocks inFIG. 2A merely illustrate an example of a preferred embodiment of thesurveillance camera according to the present invention. For example, thesurveillance camera may include an audio input unit. That is, variouschanges and modifications may be made within the scope of the presentinvention. The description of FIG. 2B will be given later on.

The names and contents of commands and parameters used in the presentembodiment will now be described.

FIG. 3 illustrates a structure of parameters held in the storage unit1002 by the surveillance camera 1000 of the present embodiment.

A MediaProfile 6100 is a parameter set for associating and storingvarious setting items of the surveillance camera. The MediaProfile 6100holds a ProfileToken which is ID of the MediaProfile 6100. Additionally,the MediaProfile 6100 holds a VideoSourceConfiguration 6102, aVideoEncoderConfiguration 6103, and a PTZConfiguration 6104. TheMediaProfile 6100 also holds links to various setting items including adelivery image encoder and an audio encoder.

A VideoSource 6101 is a group of parameters indicating performance of animage pickup sensor of the surveillance camera. The VideoSource 6101includes a VideoSourceToken which is ID of the VideoSource 6101, and aResolution indicating a resolution of image data that can be output bythe image pickup sensor. The Resolution includes a Height and a Widththat express a height and a width, respectively, in pixels.

The VideoSourceConfiguration 6102 is a group of parameters thatassociate the VideoSource 6101 with the MediaProfile 6100 of thesurveillance camera. The storage unit 1002 holds a plurality of videosource configurations, each including a resolution of a picked-up imageand used to generate the picked-up image. The VideoSourceConfiguration6102 includes a VideoSourceConfigurationToken which is ID of theVideoSourceConfiguration 6102. The VideoSourceConfigurationToken isidentification information for identifying a video source configuration.The VideoSourceConfiguration 6102 includes Bounds for designating whichpart of image data output from the VideoSource 6101 is to be cut out andused as a delivery image. The Bounds include a Height and a Width thatexpress a height and a width, respectively, of the delivery image to becut out of the image data in pixels; and x and y that expresscoordinates of a lower-left point of the delivery image to be cut out inpixels, with an upper-left point of the image data as the origin.Delivery image position information (position of the Bounds) indicatingthe position of the delivery image in the picked-up image is associatedwith identification information (VideoSourceConfigurationToken) foridentifying the delivery image position information, and a plurality ofsets of this associated information are stored. Thus, the storage unit1002 holds a plurality of video source configurations, each including areference to a configuration of a resolution of a picked-up image and areference to a configuration of a cutout position for cutting out adelivery image to be transmitted to the receiving device from thepicked-up image having the resolution.

The VideoEncoderConfiguration 6103 is a group of parameters thatassociate a configuration related to video compression with theMediaProfile 6100. The surveillance camera 1000 delivers image datawhich is output on the basis of the contents of theVideoSourceConfiguration 6102 to the client device 2000. This deliveryis according to a video compression technique (e.g., JPEG or H.264), aframe rate, or a parameter such as a resolution, set in theVideoEncoderConfiguration 6103.

The PTZConfiguration 6104 is a group of parameters that associate aconfiguration related to the pan mechanism 1101, the tilt mechanism1102, and the zoom mechanism 1103 of the surveillance camera 1000 withthe MediaProfile 6100. The PTZConfiguration 6104 includes informationabout a coordinate system that expresses actual pan and tilt anglevalues and a zoom factor value in the pan mechanism, the tilt mechanism,and the zoom mechanism. In the present example, the coordinate systemincludes angle coordinates having a pan range x=−180 to +180 (degrees)and a tilt range y=−90 to +90 (degrees), and a magnification coordinatehaving a zoom z=1 to 100 (times).

A PrivacyMask 6105 is a group of parameters that hold a list ofparameters related to a mask. The PrivacyMask 6105 includes aPrivacyMaskToken which is ID of the PrivacyMask 6105, a Name which is aname of a PrivacyMask structure, and a Mask that designates the positionand size of the mask using a polygon with three or more points. ThePrivacyMask 6105 further includes an Enabled which is a flag thatdesignates whether to display the mask on a delivery image, a Color thatdesignates the color of the mask, and a PTZDesignation that identifiesthe position of the camera platform.

The PTZDesignation includes a PTZConfigurationToken that designates thePTZConfiguration 6104 including a definition of a coordinate system forexpressing the orientation (pan and tilt) of the camera platform and thezoom factor. The PTZDesignation further includes a Position thatdesignates the orientation of the camera platform and the zoom factor inthe coordinate system indicated by the designated PTZConfiguration. ThePosition includes the orientation of the image pickup unit 1003, thatis, coordinates of the center point of image data output by the imagepickup unit and a numerical value of the zoom factor.

FIG. 4 illustrates a typical command sequence from the start of settingto the video delivery, between the surveillance camera 1000 and theclient device 2000 according to the present embodiment.

Reference numeral 7100 denotes a transaction of aGetVideoSourceConfigurations command. With this command, the clientdevice 2000 acquires a list of the VideoSourceConfiguration 6102 held bythe surveillance camera 1000.

Reference numeral 7101 denotes a transaction of aGetVideoEncoderConfigurations command. With this command, the clientdevice 2000 acquires a list of the VideoEncoderConfiguration 6103 heldby the surveillance camera 1000.

Reference numeral 7102 denotes a transaction of a GetConfigurationscommand. With this command, the client device 2000 acquires a list ofthe PTZConfiguration 6104 held by the surveillance camera 1000.

Reference numeral 7103 denotes a transaction of a CreateProfile command.With this command, the client device 2000 creates a new MediaProfile6100 in the surveillance camera 1000 and acquires its ProfileToken.

Reference numerals 7104, 7105, and 7106 denote transactions of anAddVideoSourceConfiguration command, an AdddVideoEncoderConfigurtioncommand, and an AddPTZConfiguration command, respectively. With thesecommands, the client device 2000 associates the following three, thatis, a VideoSourceConfiguration, a VideoEncoderConfiguration, and aPTZConfiguration desired for a designated MediaProfile.

Reference numeral 7107 denotes a transaction of a GetStreamUri command.With this command, the client device 2000 acquires, on the basis of theconfiguration of the designated MediaProfile, an address (URI) toacquire a delivery stream from the surveillance camera 1000.

Reference numeral 7108 denotes a transaction of a Describe command. Byexecuting this command using the URI acquired in 7107, the client device2000 requests and acquires information about contents to bestream-delivered by the surveillance camera 1000.

Reference numeral 7109 denotes a transaction of a Setup command. Byexecuting this command using the URI acquired in 7107, a method fortransmitting a stream, including a session number, is shared between theclient device 2000 and the surveillance camera 1000.

Reference numeral 7110 denotes a transaction of a Play command. Byexecuting this command using the session number acquired in 7109, theclient device 2000 requests the surveillance camera 1000 to start astream. Reference numeral 7111 denotes a delivery stream. Thesurveillance camera 1000 delivers, with the transmission method sharedin 7109, the stream that the surveillance camera 1000 has been requestedto start in 7110.

Reference numeral 7112 is a transaction of a Teardown command. Byexecuting this command using the session number acquired in 7109, theclient device 2000 requests the surveillance camera 1000 to stop thestream.

FIG. 5 illustrates a typical command sequence for mask setting betweenthe surveillance camera 1000 and the client device 2000 according to thepresent embodiment.

Reference numeral 7200 denotes a transaction of a GetPrivacyMaskscommand.

The GetPrivacyMasks command is a command used by the client device 2000to instruct the surveillance camera 1000 to return the followingPrivacyMask 6105. That is, what is returned is every PrivacyMask 6105associated with the designated VideoSourceConfiguration 6102 illustratedin FIG. 3.

Reference numeral 7201 denotes a transaction of a CreatePrivacyMaskcommand. The CreatePrivacyMask command is a command used by the clientdevice 2000 to instruct the surveillance camera 1000 to create aPrivacyMask 6105. The PrivacyMask 6105 is created in association withthe designated VideoSourceConfiguration 6102. By executing theCreatePrivacyMask command, the surveillance camera 1000 returns aPrivacyMaskToken of the created PrivacyMask 6105 to the client device2000.

Reference numeral 7202 denotes a transaction of GetPrivacyMaskOptionscommand. The GetPrivacyMaskOptions command is a command used by theclient device 2000 to instruct the surveillance camera 1000 to return aselected range or option of each parameter of the PrivacyMask 6105.Specifically, the surveillance camera 1000 is instructed to return aselected range or option of each parameter of the PrivacyMask 6105 thatcan be set by a ModifyPrivacyMask command described below.

Reference numeral 7203 denotes a transaction of a GetPrivacyMaskcommand. The GetPrivacyMask command is a command that instructs aPrivacyMask 6105 having a designated PrivacyMaskToken to be returned.The PrivacyMask 6105 has the designated PrivacyMaskToken with which theclient device 2000 is associated with the designatedVideoSourceConfiguration 6102 of the surveillance camera 1000.

Reference numeral 7204 denotes a transaction of a ModifyPrivacyMaskcommand. The ModifyPrivacyMask command is a command used by the clientdevice 2000 to instruct the surveillance camera 1000 to edit eachparameter included in the PrivacyMask 6105. By executing theModifyPrivacyMask command, the surveillance camera 1000 edits thecontent of the PrivacyMask 6105 having the PrivacyMaskToken designatedby the client device 2000. By this editing, a new mask is displayed orthe currently displayed mask is changed in color, size or position.Detailed processing of this command in the surveillance camera 1000 willbe specifically described later on.

Reference numeral 7205 denotes a transaction of a DeletePrivacyMaskcommand. The DeletePrivacyMask command is a command used by the clientdevice 2000 to instruct the surveillance camera 1000 to delete thePrivacyMask 6105 created in response to the CreatePrivacyMask command.By executing the DeletePrivacyMask command, the surveillance camera 1000deletes the PrivacyMask 6105 having the PrivacyMaskToken designated bythe client device 2000 from the storage unit.

FIG. 6 illustrates a processing procedure performed when thesurveillance camera 1000 receives the ModifyPrivacyMask command from theclient device 2000. This procedure represents part of a program readfrom the storage unit 1002 and executed by the control unit 1001, whichis a computer. The storage unit 1002 is a storage medium that storesthis program such that the control unit 1001, which is a computer, canread and execute the program.

In step S1000, the control unit 1001 determines whether a PrivacyMask6105 having a PrivacyMaskToken designated by a received command existsin the storage unit 1002. If this PrivacyMask 6105 exists, the controlunit 1001 advances the processing to step S1060.

In step S1001, the control unit 1001 checks whether a Mask included inthe PrivacyMask 6105 is correctly set. For example, the control unit1001 determines whether the Mask includes three or more points, andwhether each point is plotted within normalized coordinate system inBounds shown in 5002 of FIGS. 8A and 8B. If the Mask is not correctlyset, the control unit 1001 advances the processing to step S1060.

In step S1002, the control unit 1001 determines whether aVideoSourceConfiguration designated in the received command exists inthe storage unit 1002. If the VideoSourceConfiguration does not exist,the control unit 1001 advances the processing to step S1060.

In step S1003, the control unit 1001 causes the storage unit 1002 tostore the PrivacyMask designated in the received command.

In step S1004, the control unit 1001 reads Bounds from theVideoSourceConfiguration 6102 including a VideoSourceConfigurationTokendesignated by the client device 2000. The VideoSourceConfigurationTokenis designated by the client device 2000 memorized in the storage unit1002. That is, the surveillance camera 1000 receives identificationinformation (VideoSourceConfigurationToken) for identifying a videosource configuration from the client device 2000.

In step S1005, the control unit 1001 reads a Resolution from theVideoSource 6101 linked to (or associated with) theVideoSourceConfiguration 6102 designated in the received command. ThisVideoSourceConfiguration 6102 includes the VideoSourceConfigurationTokendesignated in the received command.

In step S1008, the control unit 1001 determines whether a PTZDesigntionis designated or omitted in the received command. If designated, thecontrol unit 1001 advances the processing to step S1009, whereas ifomitted, the control unit 1001 advances the processing to step S1050.This PTZDesigntion is image pickup direction information indicating animage pickup direction. Therefore, in step S1008, the control unit 1001determines whether image pickup direction information indicating animage pickup direction is included in mask parameters included in thereceived command.

In step S1009, the control unit 1001 reads a pan, tilt, and zoomcoordinate system from the PTZConfiguration 6104 identified by thePTZConfigurationToken designated in the PTZDesignation. On the basis ofthis coordinate system, the control unit 1001 transforms a pan value, atilt value, and a zoom value included in a Position designated in thePTZDesignation to a pan angle value, a tilt angle value, and a zoomfactor value.

In step S1050, the control unit 1001 acquires the current pan anglevalue, tilt angle value, and zoom factor value from the image pickupcontrol unit 1006.

In step S1011, the control unit 1001 executes mask display processing(the detail is omitted). The mask display processing uses the followingthree as arguments. Specifically, the first argument is the Boundsacquired in step S1004. The second argument is the Resolution acquiredin step S1005. The third argument is a set of the pan angle value, tiltangle value, and zoom value acquired in step S1009 or step S1050, andthe state of an Enabled flag included in the received command.

If YES in step S1008, the position of a mask image in an image pickuprange where the image pickup unit 1003 can pick up images is identifiedon the basis of the following three. The first is the PTZDesigntion(image pickup direction information) included in the received maskparameters. The second is delivery image position information (positionof the Bounds) associated with the VideoSourceConfigurationTokenincluded in the received mask parameters. The third is superimpositionposition information (position of the mask image) included in thereceived mask parameters. Note that the VideoSourceConfigurationToken isidentification information for identifying the delivery image positioninformation (position of the Bounds) indicating the position of thedelivery image in the picked-up image. Thus, the PTZDesignationdesignates the image pickup direction of the image pickup unit 1003 inrelation to superimposition of the mask image on the picked-up image. Asdescribed above, when a designation of the image pickup direction isreceived, the position of superimposition of the mask image in the imagepickup range where the image pickup unit 1003 can pick up images is seton the basis of the designated image pickup direction.

If NO in step S1008, the position of the mask image in the picked-upimage is identified on the basis of the following three. The first isdelivery image position information (position of the Bounds) associatedwith the VideoSourceConfigurationToken included in the received maskparameters. The second is superimposition position information (positionof the mask image) included in the received mask parameters. The thirdis the current image pickup direction of the image pickup apparatusacquired from the image pickup control unit 1006 in step S1050. Asdescribed above, when a designation of the image pickup direction of theimage pickup unit 1003 is not received in relation to superimposition ofthe mask image, the control unit 1001 sets the position ofsuperimposition of the mask image in the image pickup range where theimage pickup unit 1003 can pick up images, on the basis of the imagepickup direction identified by the image pickup control unit 1006.

In the mask display processing, the control unit 1001 inputs thearguments received in step S1011 into the mask processing unit 1007. Aswill be described with reference to FIGS. 8A and 8B and FIG. 9, the maskprocessing unit 1007 uniquely identifies the mask position in an imagepickup field of view on the basis of the input data, and applies a maskto the image being delivered on the basis of the designation of theEnabled flag. The detail of this processing will be described later on.

In step S1012, the control unit 1001 executes normal responsetransmission (the detail is omitted). In the normal responsetransmission, the control unit 1001 transmits a normal response to thereceived command via the communication unit 1004 to the client device2000.

In step S1060, the control unit 1001 executes error responsetransmission (the detail is omitted). In the error responsetransmission, the control unit 1001 transmits an error response to thereceived command via the communication unit 1004 to the client device2000.

FIGS. 8A and 8B and FIG. 9 illustrate a method for identifyingcoordinates of a mask. The operation in step S1011 will be describedwith reference to FIGS. 8A and 8B and FIG. 9.

FIG. 8A illustrates a method for designating coordinates of a Mask in aPrivacyMask. The Mask is designated with three or more points plotted ina normalized coordinate system within a delivery image 5002 cut outusing Bounds from image data output by the VideoSource 6101. Thesepoints are plotted in the normalized coordinate system where values inboth the horizontal direction (x) and the vertical direction (y) arenormalized to the range of −1.0 to +1.0, with the rightward and upwarddirections each defined as a positive direction. For example, a Mask5004 in FIG. 8A is designated with points 1 to 4 at (x,y)=(0.2,0.8),(0.8,0.8), (0.2,0.1), and (0.8,0.1).

FIG. 8B illustrates image data 5001 output from the VideoSource 6101,and the Mask 5004 superimposed on the image data 5001, the Mask 5004being described with reference to FIG. 8A. For example, assume thatResolution.Height is 960 pixels and Resolution.Width is 1280 pixels.Also assume that Bounds.Height is 480 pixels and Bounds.Width is 640pixels, and that Bounds.x is 500 pixels and Bounds.y is 900 pixels. Inthis case, to calculate the position (x2,y2) of each point of the Mask5004 in a Resolution pixel coordinate system where the center point ofthe image data 5001 is the origin, computational expressions to be usedare as follows.Mask.x2=((Bounds.x−Resolution.Width/2)+Bounds.Width/2)+Bounds.Width/2*Mask.xMask.y2=((Bounds.y−Resolution.Height/2)−Bounds.Height/2)+Bounds.Height/2*Mask.y

In the case of an image pickup apparatus that does not support thefunction of cutting out a delivery image from image data, the equation,Bounds=Resolution, is true and the computational expressions describedabove can be expressed as follows.Mask.x2=Resolution.Width/2*Mask.xMask.y2=Resolution.Height/2*Mask.yFor example, the Mask 5004 in FIG. 8B is expressed with four points at(x2,y2)=(244,372), (436,372), (436,204), and (244,204).

FIG. 9 illustrates the entire range 5000 where the image pickup unit1003 can pick up images, and the image data 5001 superimposed on theentire range 5000 on the basis of the orientation in the image pickupdirection (pan=x, tilt=y) and the zoom factor (z) designated in thePosition, the image data 5001 being illustrated in FIG. 8B. Theorientation in the image pickup direction (x,y) indicates the centerpoint of the image data 5001. A coordinate system included in thePTZConfiguration is used as a Position coordinate system. In the presentexample, the coordinate system includes angle coordinates having a panrange x=−180 to +180 (degrees) and a tilt range y=−90 to +90 (degrees),and a magnification coordinate having a zoom z=1 to 100 (times). Forexample, assume that Position.x=−20 (degrees), Position.y=+10 (degrees),and Position.z=1.2 (times). In this case, to calculate the position(x3,y3) of each point of the Mask 5004 in the coordinate system shown inthe PTZConfiguration, with the center point of the entire range 5000where the image pickup unit 1003 can pick up images as the origin,computational expressions to be used are as follows.Mask.x3=Position.x+Mask.x2/Resolution.Width/2/Position.Zoom*pan angletransformation coefficientMask.y3=Position.y+Mask.y2/Resolution.Height/2/Position.Zoom*tilt angletransformation coefficient

A zoom and pan angle transformation coefficient and a zoom and tiltangle transformation coefficient are coefficients for transforming x andy positions of the mask in the image data into pan and tilt anglecoordinates, the coefficients being based on the characteristics of theimage pickup unit 1003. In the present example, the zoom and pan angletransformation coefficient and the zoom and tilt angle transformationcoefficient are 90 and 180, respectively. The Mask 5004 in FIG. 9 isexpressed with four points at (x3,y3)=(38.19,68.13), (82.19,68.13),(82.19,41.88), and (38.19,41.88).

A flow of calculation has been described with reference to FIGS. 8A and8B and FIG. 9. The mask position (x,y) designated by coordinates in thedelivery image cut out using Bounds can be transformed into the position(x3,y3) designated by coordinates in the image pickup field of view ofthe image pickup apparatus. This transformation is performed on thebasis of information of the Bounds, Resolution, and Position. That is,the surveillance camera 1000 can uniquely identify the mask position inthe image pickup field of view. On the basis of the video sourceconfiguration identified by received identification information and theidentified image pickup direction, the control unit 1001 sets theposition of superimposition of the mask image in the image pickup rangewhere the image pickup unit 1003 can pick up images.

With the mask setting processing of the present embodiment describedabove, the following effects can be achieved. In the present embodiment,the client can designate the image pickup direction (e.g., pan and tiltpositions) of the camera in relation to the mask setting. Thus, whensetting a capturing prohibited area for each of picked-up imagesobtained in different image pickup directions, the client can designatea mask position without actually changing the image pickup direction ofthe camera. Therefore, it is possible to simultaneously performprocessing for setting capturing prohibited areas for respectivepicked-up images obtained in different image pickup directions.

The surveillance camera 1000 can uniquely identify the mask position setby the client device, regardless of having the function of cutting out,using Bounds, a specific part of image data generated by the imagepickup unit 1003 and delivering the resulting image. Thus, the imagepickup apparatus can allow the client device to set a common maskprocessing function regardless of whether Bounds are supported.

Second Embodiment

In the first embodiment, an embodiment of the present invention has beendescribed by referring to a surveillance camera in which a PrivacyMask,which is a parameter for a mask processing function, is associated witha VideoSourceConfiguration and held.

In the first embodiment, a surveillance camera has been described as anexample, which is capable of changing the image pickup range of theimage pickup unit by varying the pan, tilt, or zoom. However, thepresent invention is not limited to this. The present invention is alsoapplicable to a surveillance camera not having the pan, tilt, and zoom.

A second embodiment of the present invention that takes the above intoaccount will now be described. Note that the description of the sameparts as those of the first embodiment will be omitted.

FIG. 1B illustrates a surveillance camera 1200 according to the secondembodiment.

FIG. 2B illustrates an internal configuration of the surveillance camera1200.

FIG. 4 illustrates a typical command sequence from the start of settingto the video delivery, between the surveillance camera 1200 and theclient device 2000 according to the present embodiment.

FIG. 7 illustrates processing performed when the surveillance camera1200 receives the ModifyPrivacyMask command described above from theclient device 2000.

In step S2011, the control unit 1001 executes mask display processing byusing the Bounds acquired in step S1004, the Resolution acquired in stepS1005, and the state of the Enabled flag included in the receivedcommand as arguments. The detail of the mask display processing will beomitted.

In the mask display processing, the control unit 1001 inputs thearguments received in step S2011 into the mask processing unit 1007. Asdescribed with reference to FIGS. 8A and 8B and FIG. 9, the maskprocessing unit 1007 uniquely identifies the mask position in the imagepickup field of view on the basis of the input data, and applies a maskto the image being delivered on the basis of the designation of theEnabled flag.

The operation in step S2011 will be described with reference to FIGS. 8Aand 8B.

Since the surveillance camera 1200 cannot change the image pickup rangeof the image pickup unit 1003, the image data 5001 in FIG. 8B isequivalent to the image pickup field of view. Therefore, by the flow ofcalculation described with reference to FIGS. 8A and 8B, the maskposition (x,y) designated by coordinates in the delivery image cut outusing the Bounds can be transformed into the position (x2,y2) designatedby coordinates in the image pickup field of view of the image pickupapparatus. This transformation is performed on the basis of informationof the Bounds and Resolution. That is, the surveillance camera 1200 canuniquely identify the mask position in the image pickup field of view.Specifically, when the image pickup unit 1003 does not have the functionof changing the image pickup direction, the position of the mask imagein the image pickup range where the image pickup unit 1003 can pick upimages is identified on the basis of the following two. The first isdelivery image position information (position of the Bounds) associatedwith the VideoSourceConfigurationToken included in the received maskparameters. The second is superimposition position information (positionof the mask image) included in the received mask parameters. Note thatthe VideoSourceConfigurationToken is identification information foridentifying the delivery image position information (position of theBounds) indicating the position of the delivery image in the picked-upimage.

With the mask setting processing of the present embodiment describedabove, the surveillance camera can have a common mask processingfunction regardless of whether it has the function of changing the imagepickup range of the image pickup unit 1003 including the pan, tilt, andzoom. Thus, regardless of whether the PTZConfiguration and thePTZDesignation including the Position are supported, the image pickupapparatus is capable of allowing the client device to set a common maskprocessing function.

Although the operations of the surveillance cameras that implement thepresent invention have been described in the first and secondembodiments, the embodiments are not necessarily limited to thosedescribed above and may be partially changed as follows.

(1) The surveillance camera 1000 according to one of the embodimentsuses angle coordinates for a pan and a tilt, and also uses amagnification coordinate as a zoom coordinate in the PTZConfiguration.Alternatively, for example, normalized coordinates may be used, in whichthe entire range is normalized to the range of −1.0 to +1.0.

(2) The change of the image pickup direction in the image pickupapparatus of each embodiment has been described by using a platformdevice or the like as an example. Alternatively, a so-called electronicPTZ may be used, which is realized by electronically cutting out a partof the picked-up image.

(3) The computational expressions for uniquely identifying a maskposition have been described with reference to FIGS. 8A and 8B and FIG.9. However, the method for identifying a mask position varies dependingon the algorithm or specifications of the mask processing unit 1007 ofeach model. The type of information necessary to identify a maskposition or a computational method for identification does notnecessarily have to be those described in the embodiments.

Other Embodiments

Embodiments of the present invention may be realized by a computer of asystem or apparatus that reads out and executes computer-executableinstructions recorded on a recording medium (e.g., computer-readablenon-transitory storage device). The functions of one or more of theabove-described embodiments of the present invention are realized whenthe computer executes the instructions.

Embodiments of the present invention may be realized by a methodexecuted by the computer of the system or apparatus. For example, thefunctions of one or more of the above-described embodiments of thepresent invention are realized when the computer reads out thecomputer-executable instructions from the recording medium.

The computer may include one or more central processing units (CPUs),micro processing units (MPUs), or other circuits. The computer mayinclude a network of separate computers or separate computer processors.

The computer-executable instructions are provided to the computer, forexample, via a network or from a recording medium. Examples of therecording medium include one or more hard disks, random access memories(RAMs), read only memories (ROMs), storages of distributed computingsystems, optical disks (such as compact discs (CDs), digital versatilediscs (DVDs), or Blu-ray Discs (BDs) (registered trademark)), flashmemories, and memory cards.

The present invention can ease the complicated process of designating acoordinate position of a mask image in a client.

While the present invention has been described with reference toexemplary embodiments, it is to be understood that the invention is notlimited to the disclosed exemplary embodiments. The scope of thefollowing claims is to be accorded the broadest interpretation so as toencompass all such modifications and equivalent structures andfunctions.

The invention claimed is:
 1. An image pickup apparatus comprising: animage pickup unit capable of changing an image pickup direction to pickup an image; a receiving unit configured to receive a command includinga normalized position for setting a mask image to be displayed on thepicked up image; a determination unit configured to determine whether ornot the received command further includes a designation of the imagepickup direction of the image pickup unit for setting a position of themask image to be displayed on the picked-up image; and a setting unitconfigured to set the position of the mask image to be displayed on thepicked up image on the basis of the normalized position, the position ofthe mask image being set relative to the designated image pickupdirection of the image pickup unit in the received command if it isdetermined by the determination unit that the received command includesthe designation of the image pickup direction and to set the position ofthe mask image to be displayed on the picked up image on the basis ofthe normalized position in the received command, the position of themask image being set relative to a current image pickup direction of theimage pickup unit if it is determined by the determination unit that thereceived command does not include the designation of the image pickupdirection.
 2. The image pickup apparatus according to claim 1, furthercomprising a holding unit configured to hold a plurality of video sourceconfigurations, each including a resolution of the picked-up image andused to generate the picked-up image, wherein the receiving unitreceives identification information for identifying a video sourceconfiguration from the receiving device; and if it is determined by thedetermination unit that the received command does not include thedesignation of the image pickup direction, the setting unit sets theposition of the mask image to be displayed on the picked up image on thebasis of the video source configuration identified by the receivedidentification information and the current image pickup direction. 3.The image pickup apparatus according to claim 1, further comprising aholding unit configured to hold a plurality of configurations, eachindicating a cutout position for cutting out a delivery image to betransmitted to the receiving device from the picked-up image picked upby the image pickup unit, wherein the receiving unit receivesidentification information for identifying a configuration of the cutoutposition and position information indicating a position where the maskimage is displayed on the delivery image; and if it is determined by thedetermination unit that the received command does not include thedesignation of the image pickup direction, the setting unit sets theposition of the mask image to be displayed on the picked up image on thebasis of the configuration of the cutout position identified by thereceived identification information, the position information, and thecurrent image pickup direction.
 4. The image pickup apparatus accordingto claim 3, wherein the holding unit holds a plurality of video sourceconfigurations, each including a reference to a configuration of aresolution of the picked-up image and a reference to a configuration ofa cutout position for cutting out a delivery image to be transmitted tothe receiving device from the picked-up image having the resolution; andif it is determined by the determination unit that the received commanddoes not include the designation of the image pickup direction, thesetting unit sets the position of the mask image to be displayed on thepicked up image on the basis of the video source configurationidentified by the received identification information, the positioninformation, and the current image pickup direction.
 5. The image pickupapparatus according to claim 1, wherein the designation of the imagepickup direction of the image pickup unit includes a pan angle value, atilt angle value, and a zoom factor value.
 6. The image pickup apparatusaccording to claim 1, wherein the received command includes a coordinateof the mask image in a normalized coordinate system.
 7. A control methodfor controlling an image pickup apparatus, the control methodcomprising: a changing step for changing an image pickup direction topick up an image picked by an image pickup unit; a receiving step forreceiving a command including a normalized position for setting a maskimage to be displayed on the picked up image; a determination step fordetermining whether or not the received command further includes adesignation of the image pickup direction of the image pickup unit forsetting a position of the mask image to be displayed on the picked-upimage; and a setting step for setting the position of the mask image tobe displayed on the picked up image on the basis of the normalizedposition, the position of the mask image being set relative to thedesignated image pickup direction of the image pickup unit in thereceived command if it is determined that the received command includesthe designation of the image pickup direction and to set the position ofthe mask image to be displayed on the picked up image on the basis ofthe normalized position in the received command, the position of themask image being set relative to a current image pickup direction of theimage pickup unit if it is determined that the received command does notinclude the designation of the image pickup direction.
 8. The controlmethod according to claim 7, wherein identification information foridentifying a video source configuration including a resolution of thepicked-up image and used to generate the picked-up image is receivedfrom the receiving device in the receiving step; and if it is determinedthat the received command does not include the designation of the imagepickup direction, the position of the mask image to be displayed on thepicked up image is set, in the setting step, on the basis of the videosource configuration identified by the received identificationinformation and the current image pickup direction.
 9. The controlmethod according to claim 7, wherein identification information foridentifying a configuration of a cutout position for cutting out adelivery image to be transmitted to the receiving device from thepicked-up image picked up by the image pickup unit, and positioninformation indicating a position where the mask image is displayed onthe delivery image, are received in the receiving step; and if it isdetermined that the received command does not include the designation ofthe image pickup direction, the position of the mask image to bedisplayed on the picked up image is set in the setting step on the basisof the configuration of the cutout position identified by the receivedidentification information, the position information, and the currentimage pickup direction.
 10. The control method according to claim 8,wherein the video source configuration includes a reference to aresolution of the picked-up image and a reference to a cutout positionfor cutting out a delivery image to be transmitted to the receivingdevice from the picked-up image having the resolution; and if it isdetermined that the received command does not include the designation ofthe image pickup direction, the position of the mask image to bedisplayed on the pickup image is set in the setting step on the basis ofthe video source configuration identified by the received identificationinformation, the position information, and the current image pickupdirection.
 11. A recording medium that records a program for controllingan image pickup apparatus, the program configured to cause the imagepickup apparatus to execute: a changing step for changing an imagepickup direction of an image pickup unit to pick up an image; areceiving step for receiving a command including a normalized positionfor setting a mask image to be displayed on the picked up image; adetermination step for determining whether or not the received commandfurther includes a designation of the image pickup direction of theimage pickup unit for setting a position of the mask image to bedisplayed on the picked-up image; and a setting step for setting, theposition of the mask image to be displayed on the picked up image on thebasis of the normalized position, the position of the mask image beingset relative to the designated image pickup direction of the imagepickup unit in the received command if it is determined that thereceived command includes the designation of the image pickup directionand to set the position of the mask image to be displayed on the pickedup image on the basis of the normalized position in the receivedcommand, the position being set relative to a current image pickupdirection of the image pickup unit if it is determined that the receivedcommand does not include the designation of the image pickup direction,wherein the program causes the image pickup apparatus to executeprocessing for receiving, from the receiving device in the receivingprocedure, identification information for identifying a video sourceconfiguration including a resolution of the picked-up image and used togenerate the picked-up image; and if it is determined that the receivedcommand does not include the designation of the image pickup direction,the program causes the image pickup apparatus to execute processing forsetting, in the setting procedure, the position of the mask image to bedisplayed on the picked up image on the basis of the video sourceconfiguration identified by the received identification information andthe current image pickup direction.
 12. The recording medium accordingto claim 11, wherein the program causes the image pickup apparatus toexecute processing for receiving, in the receiving procedure,identification information for identifying a configuration of a cutoutposition for cutting out a delivery image to be transmitted to thereceiving device from the picked-up image picked up by the image pickupunit, and position information indicating a position where the maskimage is displayed on the delivery image; and if it is determined thatthe received command does not include the designation of the imagepickup direction, the program causes the image pickup apparatus toexecute processing for setting, in the setting procedure, the positionof the mask image to be displayed on the picked up image on the basis ofthe configuration of the cutout position identified by the receivedidentification information, the position information, and the currentimage pickup direction.
 13. The recording medium according to claim 11,wherein the video source configuration includes a reference to aresolution of the picked-up image and a reference to a cutout positionfor cutting out a delivery image to be transmitted to the receivingdevice from the picked-up image having the resolution; and if it isdetermined that the received command does not include the designation ofthe image pickup direction, the program causes the image pickupapparatus to execute processing for setting, in the setting procedure,the position of the mask image to be displayed on the pickup image onthe basis of the video source configuration identified by the receivedidentification information, the position information, and the currentimage pickup direction.